Performance regressions may routinely occur in large, complex applications and systems. Preventing, identifying, and remediating performance regressions may be a critical concern for application developers. Some performance regressions, such as increased usage of computing resources (e.g., processing resources, network resources, memory resources, storage resources, etc.), can affect operational costs. Additionally, some performance regressions, such as increased latency of an application (e.g., an increased start-up time, a decrease in responsiveness to user input, etc.) can degrade user experiences.
However, pinpointing root causes of performance regressions may be an arduous task, compounded by the scale and complexity of modern applications and computing systems. For example, consider preventing the start-up latency of an ANDROID application from exceeding 10 seconds. A performance regression may occur due to any of a number of factors including, but not limited to, (1) source code changes, (2) gradual roll-outs of new features, and/or (3) A/B tests of experimental features.
A complicating factor may be that a performance regression in one portion of an application may be masked by a performance improvement in another portion of the application. Another complicating factor may be that hardware and software systems that execute the application may be heterogeneous, with vastly different performance characteristics. For example, there are over 1,000 different mobile devices that run the ANDROID operating system (OS), with four major OS versions and JAVA virtual machines (JVMs). Additionally, many applications may utilize various back-end services as part of large-scale distributed systems, which may increase the difficulty of pinpointing root causes of performance regressions within and/or attributable to the applications themselves.
The instant disclosure, therefore, identifies and addresses a need for improved systems and methods for evaluating application performance to identify and determine root causes of performance regressions.